Modeling Purposive Legal Argumentation and Case Outcome Prediction using Argument Schemes�in the Value Judgment Formalism

Artificial Intelligence and Law studies how legal reasoning can be formalized in order to eventually be able to develop systems that assist lawyers in the task of researching, drafting and evaluating arguments in a professional setting. To further this goal, researchers have been developing systems, which, to a limited extent, autonomously engage in legal reasoning, and argumentation on closed domains. This dissertation presents the Value Judgment Formalism and its experimental implementation in the VJAP system, which is capable of arguing about, and predicting outcomes of, a set of trade secret misappropriation cases. VJAP argues about cases by creating an argument graph for each case using a set of argument schemes. These schemes use a representation of values underlying trade secret law and effects of facts on these values. VJAP argumentatively balances effects in the given case and analogizes it to individual precedents and the value tradeoffs in those precedents. It predicts case outcomes using a confidence measure computed from the argument graph and generates textual legal arguments justifying its predictions. The confidence propagation uses quantitative weights assigned to effects of facts on values. VJAP automatically learns these weights from past cases using an iterative optimization method. The experimental evaluation shows that VJAP generates case-based legal arguments that make plausible and intelligent-appearing use of precedents to reason about a case in terms of differences and similarities to a precedent and the value tradeoffs that both contain. VJAP’s prediction performance is promising when compared to machine learning algorithms, which do not generate legal arguments. Due to the small case base, however, the assessment of prediction performance was not statistically rigorous. VJAP exhibits argumentation and prediction behavior that, to some extent, resembles phenomena in real case-based legal reasoning, such as realistically appearing citation graphs. The VJAP system and experiment demonstrate that it is possible to effectively combine symbolic knowledge and inference with quantitative confidence propagation. In AI\&Law, such systems can embrace the structure of legal reasoning and learn quantitative information about the domain from prior cases, as well as apply this information in a structurally realistic way in the context of new cases

Using Event Progression to Enhance Purposive Argumentation in the Value Judgment Formalism

ABSTRACT This paper expands on the previously published value judgment formalism. The representation of situations is enhanced by introducing event progressions similar to actions in general AI planning. Using event progressions, situations can be assessed as to what facts they contain as well as what facts may ensue with some likelihood, thereby opening up a situation space. Purposive legal argumentation can be modeled using propositions and rules controlling the likelihoods of value-laden consequences. The paper expands the formalism to cover event progressions and illustrates the functionality using an example based on Young v. Hitchens

Attack on Unfair ToS Clause Detection: A Case Study using Universal Adversarial Triggers

Recent work has demonstrated that natural language processing techniques can support consumer protection by automatically detecting unfair clauses in the Terms of Service (ToS) Agreement. This work demonstrates that transformer-based ToS analysis systems are vulnerable to adversarial attacks. We conduct experiments attacking an unfair-clause detector with universal adversarial triggers. Experiments show that a minor perturbation of the text can considerably reduce the detection performance. Moreover, to measure the detectability of the triggers, we conduct a detailed human evaluation study by collecting both answer accuracy and response time from the participants. The results show that the naturalness of the triggers remains key to tricking readers.Comment: Accepted at NLLP@EMNLP202

From Dissonance to Insights: Dissecting Disagreements in Rationale Construction for Case Outcome Classification

In legal NLP, Case Outcome Classification (COC) must not only be accurate but also trustworthy and explainable. Existing work in explainable COC has been limited to annotations by a single expert. However, it is well-known that lawyers may disagree in their assessment of case facts. We hence collect a novel dataset RAVE: Rationale Variation in ECHR1, which is obtained from two experts in the domain of international human rights law, for whom we observe weak agreement. We study their disagreements and build a two-level task-independent taxonomy, supplemented with COC-specific subcategories. To our knowledge, this is the first work in the legal NLP that focuses on human label variation. We quantitatively assess different taxonomy categories and find that disagreements mainly stem from underspecification of the legal context, which poses challenges given the typically limited granularity and noise in COC metadata. We further assess the explainablility of SOTA COC models on RAVE and observe limited agreement between models and experts. Overall, our case study reveals hitherto underappreciated complexities in creating benchmark datasets in legal NLP that revolve around identifying aspects of a case's facts supposedly relevant to its outcome.Comment: Accepted to EMNLP 202

VECHR: A Dataset for Explainable and Robust Classification of Vulnerability Type in the European Court of Human Rights

Recognizing vulnerability is crucial for understanding and implementing targeted support to empower individuals in need. This is especially important at the European Court of Human Rights (ECtHR), where the court adapts Convention standards to meet actual individual needs and thus ensures effective human rights protection. However, the concept of vulnerability remains elusive at the ECtHR and no prior NLP research has dealt with it. To enable future research in this area, we present VECHR, a novel expert-annotated multi-label dataset comprising of vulnerability type classification and explanation rationale. We benchmark the performance of state-of-the-art models on VECHR from both prediction and explainability perspectives. Our results demonstrate the challenging nature of the task with lower prediction performance and limited agreement between models and experts. Further, we analyze the robustness of these models in dealing with out-of-domain (OOD) data and observe overall limited performance. Our dataset poses unique challenges offering significant room for improvement regarding performance, explainability, and robustness.Comment: Accepted to EMNLP 202

Introduction for artificial intelligence and law: special issue “natural language processing for legal texts”

International audienc